{
  "cells": [
    {
      "cell_type": "markdown",
      "id": "1d82425a-fc89-40a4-a1f8-29d1f93f50e3",
      "metadata": {
        "id": "1d82425a-fc89-40a4-a1f8-29d1f93f50e3"
      },
      "source": [
        "\n",
        "<div align=\"center\">\n",
        "  <img src=\"https://github.com/hitsz-ids/synthetic-data-generator/blob/main/assets/sdg_logo.png?raw=true\" width=\"400\" >\n",
        "</div>\n",
        "<div align=\"center\">"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5e5e8227-e94b-4b21-b4ac-e79a28408f4b",
      "metadata": {
        "id": "5e5e8227-e94b-4b21-b4ac-e79a28408f4b"
      },
      "source": [
        "\n",
        "\n",
        "\n",
        "\n",
        "# 🚀 Synthetic data generation without Raw Data using LLM\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "The Synthetic Data Generator (SDG) is a specialized framework designed to generate high-quality structured tabular data. It incorporates a wide range of single-table, multi-table data synthesis algorithms and LLM-based synthetic data generation models.\n",
        "\n",
        "Synthetic data, generated by machines using real data, metadata, and algorithms, does not contain any sensitive information, yet it retains the essential characteristics of the original data. There is no direct correlation between synthetic data and real data, making it exempt from privacy regulations such as GDPR and ADPPA. This eliminates the risk of privacy breaches in practical applications."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b95e4903-89f4-46e0-b7a7-caecb6d179e4",
      "metadata": {
        "tags": [],
        "id": "b95e4903-89f4-46e0-b7a7-caecb6d179e4"
      },
      "outputs": [],
      "source": [
        "from sdgx.models.LLM.single_table.gpt import *"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d142956e-279b-44d7-9950-9530cd4de864",
      "metadata": {
        "tags": [],
        "id": "d142956e-279b-44d7-9950-9530cd4de864"
      },
      "outputs": [],
      "source": [
        "class SingleTableGLMModel(SingleTableGPTModel):\n",
        "\n",
        "    def ask_gpt(self, question, model=None):\n",
        "        \"\"\"\n",
        "        Sends a question to the GPT model.\n",
        "\n",
        "        Args:\n",
        "            question (str): The question to ask.\n",
        "            model (str): The GPT model to use. Defaults to None.\n",
        "\n",
        "        Returns:\n",
        "            str: The response from the GPT model.\n",
        "\n",
        "        Raises:\n",
        "            SynthesizerInitError: If the check method fails.\n",
        "        \"\"\"\n",
        "        self.check()\n",
        "        api_key = self.openai_API_key\n",
        "        if model:\n",
        "            model = model\n",
        "        else:\n",
        "            model = self.gpt_model\n",
        "        openai.api_key = api_key\n",
        "        client = openai.OpenAI(base_url=self.openai_API_url,\n",
        "                               api_key=api_key)\n",
        "        logger.info(f\"Ask GPT with temperature = {self.temperature}.\")\n",
        "        response = client.chat.completions.create(\n",
        "            model=model,\n",
        "            messages=[\n",
        "                {\n",
        "                    \"role\": \"user\",\n",
        "                    \"content\": question,\n",
        "                },\n",
        "            ],\n",
        "\n",
        "            temperature=self.temperature,\n",
        "            max_tokens=self.max_tokens,\n",
        "            timeout=self.timeout,\n",
        "        )\n",
        "        logger.info(\"Ask GPT Finished.\")\n",
        "        # store response\n",
        "        self._responses.append(response)\n",
        "        # return the content of the gpt response\n",
        "        return response.choices[0].message.content\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "0da89ca5-0be6-480a-bfc4-6a18d5c4f38c",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "0da89ca5-0be6-480a-bfc4-6a18d5c4f38c",
        "outputId": "8c2a10cc-a12f-4868-c9a7-f472ffec1523",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Requirement already satisfied: sdgx in /usr/local/lib/python3.10/dist-packages (0.2.0)\n",
            "Requirement already satisfied: click in /usr/local/lib/python3.10/dist-packages (from sdgx) (8.1.7)\n",
            "Requirement already satisfied: cloudpickle in /usr/local/lib/python3.10/dist-packages (from sdgx) (2.2.1)\n",
            "Requirement already satisfied: faker>=10 in /usr/local/lib/python3.10/dist-packages (from sdgx) (25.8.0)\n",
            "Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.10/dist-packages (from sdgx) (7.1.0)\n",
            "Requirement already satisfied: loguru in /usr/local/lib/python3.10/dist-packages (from sdgx) (0.7.2)\n",
            "Requirement already satisfied: matplotlib in /usr/local/lib/python3.10/dist-packages (from sdgx) (3.7.1)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from sdgx) (1.25.2)\n",
            "Requirement already satisfied: openai>=1.10.0 in /usr/local/lib/python3.10/dist-packages (from sdgx) (1.33.0)\n",
            "Requirement already satisfied: pandas in /usr/local/lib/python3.10/dist-packages (from sdgx) (2.0.3)\n",
            "Requirement already satisfied: pluggy in /usr/local/lib/python3.10/dist-packages (from sdgx) (1.5.0)\n",
            "Requirement already satisfied: pyarrow in /usr/local/lib/python3.10/dist-packages (from sdgx) (14.0.2)\n",
            "Requirement already satisfied: pydantic>=2 in /usr/local/lib/python3.10/dist-packages (from sdgx) (2.7.3)\n",
            "Requirement already satisfied: pyyaml in /usr/local/lib/python3.10/dist-packages (from sdgx) (6.0.1)\n",
            "Requirement already satisfied: scikit-learn<2,>=0.24 in /usr/local/lib/python3.10/dist-packages (from sdgx) (1.2.2)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from sdgx) (1.11.4)\n",
            "Requirement already satisfied: table-evaluator in /usr/local/lib/python3.10/dist-packages (from sdgx) (1.6.1)\n",
            "Requirement already satisfied: torch>=2 in /usr/local/lib/python3.10/dist-packages (from sdgx) (2.3.0+cu121)\n",
            "Requirement already satisfied: torchvision in /usr/local/lib/python3.10/dist-packages (from sdgx) (0.18.0+cu121)\n",
            "Requirement already satisfied: python-dateutil>=2.4 in /usr/local/lib/python3.10/dist-packages (from faker>=10->sdgx) (2.8.2)\n",
            "Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.10.0->sdgx) (3.7.1)\n",
            "Requirement already satisfied: distro<2,>=1.7.0 in /usr/lib/python3/dist-packages (from openai>=1.10.0->sdgx) (1.7.0)\n",
            "Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from openai>=1.10.0->sdgx) (0.27.0)\n",
            "Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from openai>=1.10.0->sdgx) (1.3.1)\n",
            "Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai>=1.10.0->sdgx) (4.66.4)\n",
            "Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from openai>=1.10.0->sdgx) (4.12.1)\n",
            "Requirement already satisfied: annotated-types>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from pydantic>=2->sdgx) (0.7.0)\n",
            "Requirement already satisfied: pydantic-core==2.18.4 in /usr/local/lib/python3.10/dist-packages (from pydantic>=2->sdgx) (2.18.4)\n",
            "Requirement already satisfied: joblib>=1.1.1 in /usr/local/lib/python3.10/dist-packages (from scikit-learn<2,>=0.24->sdgx) (1.4.2)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.10/dist-packages (from scikit-learn<2,>=0.24->sdgx) (3.5.0)\n",
            "Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (3.14.0)\n",
            "Requirement already satisfied: sympy in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (1.12.1)\n",
            "Requirement already satisfied: networkx in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (3.3)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (3.1.4)\n",
            "Requirement already satisfied: fsspec in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (2023.6.0)\n",
            "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (12.1.105)\n",
            "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (12.1.105)\n",
            "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (12.1.105)\n",
            "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (8.9.2.26)\n",
            "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (12.1.3.1)\n",
            "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (11.0.2.54)\n",
            "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (10.3.2.106)\n",
            "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (11.4.5.107)\n",
            "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (12.1.0.106)\n",
            "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (2.20.5)\n",
            "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (12.1.105)\n",
            "Requirement already satisfied: triton==2.3.0 in /usr/local/lib/python3.10/dist-packages (from torch>=2->sdgx) (2.3.0)\n",
            "Requirement already satisfied: nvidia-nvjitlink-cu12 in /usr/local/lib/python3.10/dist-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=2->sdgx) (12.5.40)\n",
            "Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.10/dist-packages (from importlib-metadata->sdgx) (3.19.1)\n",
            "Requirement already satisfied: contourpy>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (1.2.1)\n",
            "Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (0.12.1)\n",
            "Requirement already satisfied: fonttools>=4.22.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (4.53.0)\n",
            "Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (1.4.5)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (24.0)\n",
            "Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (9.4.0)\n",
            "Requirement already satisfied: pyparsing>=2.3.1 in /usr/local/lib/python3.10/dist-packages (from matplotlib->sdgx) (3.1.2)\n",
            "Requirement already satisfied: pytz>=2020.1 in /usr/local/lib/python3.10/dist-packages (from pandas->sdgx) (2023.4)\n",
            "Requirement already satisfied: tzdata>=2022.1 in /usr/local/lib/python3.10/dist-packages (from pandas->sdgx) (2024.1)\n",
            "Requirement already satisfied: psutil in /usr/local/lib/python3.10/dist-packages (from table-evaluator->sdgx) (5.9.5)\n",
            "Requirement already satisfied: dython==0.7.3 in /usr/local/lib/python3.10/dist-packages (from table-evaluator->sdgx) (0.7.3)\n",
            "Requirement already satisfied: seaborn in /usr/local/lib/python3.10/dist-packages (from table-evaluator->sdgx) (0.13.1)\n",
            "Requirement already satisfied: scikit-plot>=0.3.7 in /usr/local/lib/python3.10/dist-packages (from dython==0.7.3->table-evaluator->sdgx) (0.3.7)\n",
            "Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai>=1.10.0->sdgx) (3.7)\n",
            "Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->openai>=1.10.0->sdgx) (1.2.1)\n",
            "Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai>=1.10.0->sdgx) (2024.6.2)\n",
            "Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->openai>=1.10.0->sdgx) (1.0.5)\n",
            "Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->openai>=1.10.0->sdgx) (0.14.0)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.10/dist-packages (from python-dateutil>=2.4->faker>=10->sdgx) (1.16.0)\n",
            "Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.10/dist-packages (from jinja2->torch>=2->sdgx) (2.1.5)\n",
            "Requirement already satisfied: mpmath<1.4.0,>=1.1.0 in /usr/local/lib/python3.10/dist-packages (from sympy->torch>=2->sdgx) (1.3.0)\n"
          ]
        }
      ],
      "source": [
        "# install dependencies\n",
        "!pip install sdgx\n",
        "# OR\n",
        "# !pip install git+https://github.com/hitsz-ids/synthetic-data-generator.git"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "83aa5c17-99bf-4f50-9bd8-46af59fc9494",
      "metadata": {
        "id": "83aa5c17-99bf-4f50-9bd8-46af59fc9494"
      },
      "source": [
        "We demonstrate with a single table synthetic example.\n",
        "\n",
        "# LLM-integrated synthetic data generation\n",
        "\n",
        "For a long time, LLM has been used to understand and generate various types of data.\n",
        "\n",
        "In fact, LLM also has certain capabilities in tabular data generation. LLM has some abilities that cannot be achieved by traditional (GAN-based models or statistical models) .\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "874c0a7c-d5f0-470c-9bab-f469d134c4cc",
      "metadata": {
        "id": "874c0a7c-d5f0-470c-9bab-f469d134c4cc",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# please set your GLM4 key here:\n",
        "\n",
        "GLM4_AI_KEY = {YOUR_KEY}\n",
        "GLM4_AI_BASE = 'https://open.bigmodel.cn/api/paas/v4/'"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f5f18d39-0c56-4a14-a167-f63aa8fd20a8",
      "metadata": {
        "id": "f5f18d39-0c56-4a14-a167-f63aa8fd20a8",
        "tags": []
      },
      "outputs": [],
      "source": [
        "# import packages\n",
        "\n",
        "import pandas as pd\n",
        "from sdgx.utils import download_demo_data\n",
        "from sdgx.data_models.metadata import Metadata\n",
        "from sdgx.models.LLM.single_table.gpt import SingleTableGPTModel\n",
        "\n",
        "# read the demo data\n",
        "# currently we use the well-known adult dataset as a example\n",
        "data_path = download_demo_data()\n",
        "df = pd.read_csv(data_path)\n",
        "metadata = Metadata.from_dataframe(df)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9dc77c37-8ecd-4cf8-8cba-8b2861a25442",
      "metadata": {
        "id": "9dc77c37-8ecd-4cf8-8cba-8b2861a25442",
        "tags": []
      },
      "source": [
        "\n",
        "# Synthetic data generation without Data\n",
        "\n",
        "\n",
        "Our `sdgx.models.LLM.single_table.gpt.SingleTableGPTModel` implements “Synthetic data generation without Raw Data”.\n",
        "\n",
        "No training data is required, synthetic data can be generated based on metadata data.\n",
        "\n",
        "![LLM_Case_1](https://github.com/hitsz-ids/synthetic-data-generator/blob/main/assets/LLM_Case_1.gif?raw=true)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "221a206d-9553-4120-9b78-fa2670078b36",
      "metadata": {
        "id": "221a206d-9553-4120-9b78-fa2670078b36",
        "tags": []
      },
      "outputs": [],
      "source": [
        "model = SingleTableGLMModel()\n",
        "model.set_openAI_settings(GLM4_AI_BASE, GLM4_AI_KEY)\n",
        "model.gpt_model = \"glm-4\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f31ddfdf-6060-48dc-af9f-c74bd00ab0d4",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "f31ddfdf-6060-48dc-af9f-c74bd00ab0d4",
        "outputId": "14dfe592-3206-4aac-e3f3-34110e928530",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[32m2024-06-10 13:52:27.718\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36m_fit_with_metadata\u001b[0m:\u001b[36m228\u001b[0m - \u001b[1mFitting model with metadata...\u001b[0m\n",
            "\u001b[32m2024-06-10 13:52:27.722\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36m_fit_with_metadata\u001b[0m:\u001b[36m232\u001b[0m - \u001b[1mFitting model with metadata... Finished.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:52:27.725\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36msample\u001b[0m:\u001b[36m385\u001b[0m - \u001b[1mSampling use GPT model ...\u001b[0m\n",
            "\u001b[32m2024-06-10 13:52:27.728\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36m_sample_with_metadata\u001b[0m:\u001b[36m446\u001b[0m - \u001b[1mSampling with metadata.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:52:27.730\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.base\u001b[0m:\u001b[36m_form_dataset_description\u001b[0m:\u001b[36m122\u001b[0m - \u001b[1mNo dataset_description given in current model.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:52:27.733\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.base\u001b[0m:\u001b[36m_form_message_with_offtable_features\u001b[0m:\u001b[36m108\u001b[0m - \u001b[1mNo off_table_feature needed in current model.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:52:27.805\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36m__main__\u001b[0m:\u001b[36mask_gpt\u001b[0m:\u001b[36m26\u001b[0m - \u001b[1mAsk GPT with temperature = 0.1.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:54:34.340\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36m__main__\u001b[0m:\u001b[36mask_gpt\u001b[0m:\u001b[36m40\u001b[0m - \u001b[1mAsk GPT Finished.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:54:34.342\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36mextract_samples_from_response\u001b[0m:\u001b[36m357\u001b[0m - \u001b[1mExtracting samples from response ...\u001b[0m\n",
            "\u001b[32m2024-06-10 13:54:34.353\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36mextract_samples_from_response\u001b[0m:\u001b[36m369\u001b[0m - \u001b[1mExtracting samples from response ... Finished, 10 extracted.\u001b[0m\n",
            "\u001b[32m2024-06-10 13:54:34.357\u001b[0m | \u001b[1mINFO    \u001b[0m | \u001b[36msdgx.models.LLM.single_table.gpt\u001b[0m:\u001b[36msample\u001b[0m:\u001b[36m395\u001b[0m - \u001b[1mSampling use GPT model ... Finished.\u001b[0m\n"
          ]
        }
      ],
      "source": [
        "model.fit(metadata)\n",
        "# this may take a while\n",
        "sampled_data = model.sample(30)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f8a64c7a-3d62-4ae3-9e5a-1ed0fa6dceae",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 554
        },
        "id": "f8a64c7a-3d62-4ae3-9e5a-1ed0fa6dceae",
        "outputId": "4924004d-b619-4c62-dac2-ba8137c54f06",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "   fnlwgt capital-loss age educational-num         occupation   relationship  \\\n",
              "0  249472            0  52              13    Exec-managerial        Husband   \n",
              "1  312466         1900  31              10       Craft-repair  Not-in-family   \n",
              "2  222211            0  38              12       Adm-clerical      Own-child   \n",
              "3  198839          500  28               9      Other-service           Wife   \n",
              "4  291775            0  44              14     Prof-specialty        Husband   \n",
              "5  178556         1200  23               7  Machine-op-inspct      Own-child   \n",
              "6  275935            0  36              11       Tech-support  Not-in-family   \n",
              "7  392763         3000  29               8              Sales           Wife   \n",
              "8  234733            0  41              13    Exec-managerial        Husband   \n",
              "9  153660            0  54              15     Prof-specialty           Wife   \n",
              "\n",
              "  native-country  gender     education income         workclass  \\\n",
              "0  United-States    Male     Bachelors  <=50K           Private   \n",
              "1         Mexico  Female       HS-grad  <=50K  Self-emp-not-inc   \n",
              "2    Philippines    Male     Assoc-voc  <=50K         Local-gov   \n",
              "3        Jamaica  Female  Some-college  <=50K           Private   \n",
              "4  United-States    Male       Masters   >50K      Self-emp-inc   \n",
              "5         Canada  Female     Bachelors  <=50K           Private   \n",
              "6          India    Male    Assoc-acdm  <=50K         State-gov   \n",
              "7  United-States  Female       HS-grad  <=50K           Private   \n",
              "8  United-States    Male   Prof-school   >50K      Self-emp-inc   \n",
              "9  United-States  Female     Doctorate   >50K         Local-gov   \n",
              "\n",
              "                 race capital-gain      marital-status hours-per-week  \n",
              "0               White            0  Married-civ-spouse             40  \n",
              "1            Hispanic            0       Never-married             45  \n",
              "2  Asian-Pac-Islander         1500  Married-civ-spouse             38  \n",
              "3               Black            0   Married-AF-spouse             37  \n",
              "4               White         2500  Married-civ-spouse             50  \n",
              "5               White            0       Never-married             30  \n",
              "6  Asian-Pac-Islander            0            Divorced             40  \n",
              "7               Black            0  Married-civ-spouse             35  \n",
              "8               White         5000  Married-civ-spouse             60  \n",
              "9               White            0  Married-civ-spouse             40  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-bfcd10a6-50bd-4a96-b25c-153b05196fd6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>fnlwgt</th>\n",
              "      <th>capital-loss</th>\n",
              "      <th>age</th>\n",
              "      <th>educational-num</th>\n",
              "      <th>occupation</th>\n",
              "      <th>relationship</th>\n",
              "      <th>native-country</th>\n",
              "      <th>gender</th>\n",
              "      <th>education</th>\n",
              "      <th>income</th>\n",
              "      <th>workclass</th>\n",
              "      <th>race</th>\n",
              "      <th>capital-gain</th>\n",
              "      <th>marital-status</th>\n",
              "      <th>hours-per-week</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>249472</td>\n",
              "      <td>0</td>\n",
              "      <td>52</td>\n",
              "      <td>13</td>\n",
              "      <td>Exec-managerial</td>\n",
              "      <td>Husband</td>\n",
              "      <td>United-States</td>\n",
              "      <td>Male</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>Private</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>40</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>312466</td>\n",
              "      <td>1900</td>\n",
              "      <td>31</td>\n",
              "      <td>10</td>\n",
              "      <td>Craft-repair</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>Mexico</td>\n",
              "      <td>Female</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>Self-emp-not-inc</td>\n",
              "      <td>Hispanic</td>\n",
              "      <td>0</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>45</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>222211</td>\n",
              "      <td>0</td>\n",
              "      <td>38</td>\n",
              "      <td>12</td>\n",
              "      <td>Adm-clerical</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>Philippines</td>\n",
              "      <td>Male</td>\n",
              "      <td>Assoc-voc</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>Local-gov</td>\n",
              "      <td>Asian-Pac-Islander</td>\n",
              "      <td>1500</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>38</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>198839</td>\n",
              "      <td>500</td>\n",
              "      <td>28</td>\n",
              "      <td>9</td>\n",
              "      <td>Other-service</td>\n",
              "      <td>Wife</td>\n",
              "      <td>Jamaica</td>\n",
              "      <td>Female</td>\n",
              "      <td>Some-college</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>Private</td>\n",
              "      <td>Black</td>\n",
              "      <td>0</td>\n",
              "      <td>Married-AF-spouse</td>\n",
              "      <td>37</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>291775</td>\n",
              "      <td>0</td>\n",
              "      <td>44</td>\n",
              "      <td>14</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Husband</td>\n",
              "      <td>United-States</td>\n",
              "      <td>Male</td>\n",
              "      <td>Masters</td>\n",
              "      <td>&gt;50K</td>\n",
              "      <td>Self-emp-inc</td>\n",
              "      <td>White</td>\n",
              "      <td>2500</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>50</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>178556</td>\n",
              "      <td>1200</td>\n",
              "      <td>23</td>\n",
              "      <td>7</td>\n",
              "      <td>Machine-op-inspct</td>\n",
              "      <td>Own-child</td>\n",
              "      <td>Canada</td>\n",
              "      <td>Female</td>\n",
              "      <td>Bachelors</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>Private</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>Never-married</td>\n",
              "      <td>30</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>275935</td>\n",
              "      <td>0</td>\n",
              "      <td>36</td>\n",
              "      <td>11</td>\n",
              "      <td>Tech-support</td>\n",
              "      <td>Not-in-family</td>\n",
              "      <td>India</td>\n",
              "      <td>Male</td>\n",
              "      <td>Assoc-acdm</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>State-gov</td>\n",
              "      <td>Asian-Pac-Islander</td>\n",
              "      <td>0</td>\n",
              "      <td>Divorced</td>\n",
              "      <td>40</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>392763</td>\n",
              "      <td>3000</td>\n",
              "      <td>29</td>\n",
              "      <td>8</td>\n",
              "      <td>Sales</td>\n",
              "      <td>Wife</td>\n",
              "      <td>United-States</td>\n",
              "      <td>Female</td>\n",
              "      <td>HS-grad</td>\n",
              "      <td>&lt;=50K</td>\n",
              "      <td>Private</td>\n",
              "      <td>Black</td>\n",
              "      <td>0</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>35</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>234733</td>\n",
              "      <td>0</td>\n",
              "      <td>41</td>\n",
              "      <td>13</td>\n",
              "      <td>Exec-managerial</td>\n",
              "      <td>Husband</td>\n",
              "      <td>United-States</td>\n",
              "      <td>Male</td>\n",
              "      <td>Prof-school</td>\n",
              "      <td>&gt;50K</td>\n",
              "      <td>Self-emp-inc</td>\n",
              "      <td>White</td>\n",
              "      <td>5000</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>60</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>153660</td>\n",
              "      <td>0</td>\n",
              "      <td>54</td>\n",
              "      <td>15</td>\n",
              "      <td>Prof-specialty</td>\n",
              "      <td>Wife</td>\n",
              "      <td>United-States</td>\n",
              "      <td>Female</td>\n",
              "      <td>Doctorate</td>\n",
              "      <td>&gt;50K</td>\n",
              "      <td>Local-gov</td>\n",
              "      <td>White</td>\n",
              "      <td>0</td>\n",
              "      <td>Married-civ-spouse</td>\n",
              "      <td>40</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-bfcd10a6-50bd-4a96-b25c-153b05196fd6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-bfcd10a6-50bd-4a96-b25c-153b05196fd6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-bfcd10a6-50bd-4a96-b25c-153b05196fd6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-825777d0-4cbb-4f5d-a342-36fd97855c73\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-825777d0-4cbb-4f5d-a342-36fd97855c73')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-825777d0-4cbb-4f5d-a342-36fd97855c73 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "sampled_data",
              "summary": "{\n  \"name\": \"sampled_data\",\n  \"rows\": 10,\n  \"fields\": [\n    {\n      \"column\": \"fnlwgt\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 10,\n        \"samples\": [\n          \"234733\",\n          \"312466\",\n          \"178556\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"capital-loss\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"1900\",\n          \"3000\",\n          \"500\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"age\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 10,\n        \"samples\": [\n          \"41\",\n          \"31\",\n          \"23\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"educational-num\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9,\n        \"samples\": [\n          \"8\",\n          \"10\",\n          \"7\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"occupation\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 8,\n        \"samples\": [\n          \"Craft-repair\",\n          \"Machine-op-inspct\",\n          \"Exec-managerial\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"relationship\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"Not-in-family\",\n          \"Wife\",\n          \"Husband\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"native-country\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 6,\n        \"samples\": [\n          \"United-States\",\n          \"Mexico\",\n          \"India\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"gender\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \"Female\",\n          \"Male\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"education\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 8,\n        \"samples\": [\n          \"HS-grad\",\n          \"Assoc-acdm\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"income\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 2,\n        \"samples\": [\n          \">50K\",\n          \"<=50K\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"workclass\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 5,\n        \"samples\": [\n          \"Self-emp-not-inc\",\n          \"State-gov\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"race\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"Hispanic\",\n          \"Black\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"capital-gain\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"1500\",\n          \"5000\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"marital-status\",\n      \"properties\": {\n        \"dtype\": \"category\",\n        \"num_unique_values\": 4,\n        \"samples\": [\n          \"Never-married\",\n          \"Divorced\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"hours-per-week\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 8,\n        \"samples\": [\n          \"45\",\n          \"30\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 18
        }
      ],
      "source": [
        "sampled_data"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1a43c87a-808f-4594-abf7-11e0d1a2af2b",
      "metadata": {
        "id": "1a43c87a-808f-4594-abf7-11e0d1a2af2b"
      },
      "source": [
        "View the original information returned by gpt through the `_responses` attribute."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "729fe566-8e0a-496f-9284-f98bb39b1e8f",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "729fe566-8e0a-496f-9284-f98bb39b1e8f",
        "outputId": "c897452a-6771-45c0-97c2-7a65254157e3",
        "tags": []
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[ChatCompletion(id='8730210182522821228', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\"Here are 30 synthetic data samples based on the provided column headers:\\n\\n1. fnlwgt is 249472, capital-loss is 0, age is 52, educational-num is 13, occupation is Exec-managerial, relationship is Husband, native-country is United-States, gender is Male, education is Bachelors, income is <=50K, workclass is Private, race is White, capital-gain is 0, marital-status is Married-civ-spouse, hours-per-week is 40\\n2. fnlwgt is 312466, capital-loss is 1900, age is 31, educational-num is 10, occupation is Craft-repair, relationship is Not-in-family, native-country is Mexico, gender is Female, education is HS-grad, income is <=50K, workclass is Self-emp-not-inc, race is Hispanic, capital-gain is 0, marital-status is Never-married, hours-per-week is 45\\n3. fnlwgt is 222211, capital-loss is 0, age is 38, educational-num is 12, occupation is Adm-clerical, relationship is Own-child, native-country is Philippines, gender is Male, education is Assoc-voc, income is <=50K, workclass is Local-gov, race is Asian-Pac-Islander, capital-gain is 1500, marital-status is Married-civ-spouse, hours-per-week is 38\\n4. fnlwgt is 198839, capital-loss is 500, age is 28, educational-num is 9, occupation is Other-service, relationship is Wife, native-country is Jamaica, gender is Female, education is Some-college, income is <=50K, workclass is Private, race is Black, capital-gain is 0, marital-status is Married-AF-spouse, hours-per-week is 37\\n5. fnlwgt is 291775, capital-loss is 0, age is 44, educational-num is 14, occupation is Prof-specialty, relationship is Husband, native-country is United-States, gender is Male, education is Masters, income is >50K, workclass is Self-emp-inc, race is White, capital-gain is 2500, marital-status is Married-civ-spouse, hours-per-week is 50\\n6. fnlwgt is 178556, capital-loss is 1200, age is 23, educational-num is 7, occupation is Machine-op-inspct, relationship is Own-child, native-country is Canada, gender is Female, education is Bachelors, income is <=50K, workclass is Private, race is White, capital-gain is 0, marital-status is Never-married, hours-per-week is 30\\n7. fnlwgt is 275935, capital-loss is 0, age is 36, educational-num is 11, occupation is Tech-support, relationship is Not-in-family, native-country is India, gender is Male, education is Assoc-acdm, income is <=50K, workclass is State-gov, race is Asian-Pac-Islander, capital-gain is 0, marital-status is Divorced, hours-per-week is 40\\n8. fnlwgt is 392763, capital-loss is 3000, age is 29, educational-num is 8, occupation is Sales, relationship is Wife, native-country is United-States, gender is Female, education is HS-grad, income is <=50K, workclass is Private, race is Black, capital-gain is 0, marital-status is Married-civ-spouse, hours-per-week is 35\\n9. fnlwgt is 234733, capital-loss is 0, age is 41, educational-num is 13, occupation is Exec-managerial, relationship is Husband, native-country is United-States, gender is Male, education is Prof-school, income is >50K, workclass is Self-emp-inc, race is White, capital-gain is 5000, marital-status is Married-civ-spouse, hours-per-week is 60\\n10. fnlwgt is 153660, capital-loss is 0, age is 54, educational-num is 15, occupation is Prof-specialty, relationship is Wife, native-country is United-States, gender is Female, education is Doctorate, income is >50K, workclass is Local-gov, race is White, capital-gain is 0, marital-status is Married-civ-spouse, hours-per-week is 40\\n\\n... and so on for a total of 30 data samples. Since the format requires one sample per line, I've provided 10 samples here. If you need the remaining 20, please let me know, and I'll be happy to provide them.\", role='assistant', function_call=None, tool_calls=None))], created=1718027674, model='glm-4', object=None, system_fingerprint=None, usage=CompletionUsage(completion_tokens=1021, prompt_tokens=267, total_tokens=1288), request_id='8730210182522821228')]"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ],
      "source": [
        "model._responses"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.14"
    },
    "colab": {
      "provenance": []
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}